Methods and apparatus to monitor a split screen media presentation

ABSTRACT

Methods, apparatus, systems and articles of manufacture are disclosed to monitor a split screen media presentation. Example apparatus disclosed herein are to generate one or more video signatures associated with video output from a media device and determine whether the one or more video signatures correspond to first media identified based on audio output from the media device. Disclosed example apparatus are also to, in response to a determination that the one or more generated video signatures do not correspond to the first media, determine that the video corresponds to a split-screen presentation having a plurality of regions-of-interest, and identify a first one of the regions-of-interest of the split-screen presentation that includes the first media.

RELATED APPLICATIONS

This patent arises from a continuation of U.S. patent application Ser.No. 16/033,018, now U.S. Pat. No. ______, filed on Jul. 11, 2018.Priority to U.S. patent application Ser. No. 16/033,018 is claimed. U.S.patent application Ser. No. 16/033,018 is hereby incorporated herein byreference in its entirety.

FIELD OF THE DISCLOSURE

This disclosure relates generally to audience measurement, and, moreparticularly, to methods and apparatus to monitor a split screen mediapresentation.

BACKGROUND

Audience measurement of media (e.g., broadcast television and/or radio,stored audio and/or video content played back from a memory such as adigital video recorder or a digital video disc, a webpage, audio and/orvideo media presented (e.g., streamed) via the Internet, a video game,etc.) often involves collection of media identifying data (e.g.,signature(s), fingerprint(s), code(s), tuned channel identificationinformation, time of exposure information, etc.) and people data (e.g.,user identifiers, demographic data associated with audience members,etc.). The media identifying data and the people data can be combined togenerate, for example, media exposure data indicative of amount(s)and/or type(s) of people that were exposed to the monitored piece(s) ofmedia.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example media monitoring system coupledto an example media entertainment system and structured to monitor splitscreen media presentations in accordance with teachings of thisdisclosure.

FIG. 2A is a block diagram of an example meter that may be used toimplement the example media monitoring system of FIG. 1.

FIG. 2B is a block diagram of an example central facility that may beused to implement the example media monitoring system of FIG. 1.

FIGS. 3A-B and 4 are flowcharts representative of machine readableinstructions which may be executed to implement the example meter ofFIG. 2A.

FIG. 5 is an example flowchart representative of machine readableinstructions which may be executed to implement the example centralfacility of FIG. 2B.

FIG. 6 is a block diagram of an example processing platform structuredto execute the instructions of FIGS. 3A-B and 4 to implement the examplemeter of FIG. 2A.

FIG. 7 is a block diagram of an example processing platform structuredto execute the instructions of FIG. 5 to implement the example centralfacility of FIG. 2B.

The figures are not to scale. In general, the same reference numberswill be used throughout the drawing(s) and accompanying writtendescription to refer to the same or like parts.

DETAILED DESCRIPTION

An audience measurement entity (e.g., The Nielsen Company (US), LLC) cancalculate ratings for a first piece of media (e.g., a televisionprogram) by correlating data collected from a plurality of panelistsites with the demographics of the panelist. For example, at eachpanelist site where the first piece of media is detected in themonitored environment at a first time, media identifying information forthe first piece of media is correlated with presence informationdetected in the environment at the first time. The results from multiplepanelist sites are combined and/or analyzed to provide ratingsrepresentative of exposure of a population to the first piece of media.As used herein, the term “media” includes any type of content and/oradvertisement delivered via any type of distribution medium. Thus, mediaincludes television programming or advertisements, radio programming oradvertisements, movies, web sites, streaming media, etc.

As used herein, “broadcast” refers to any sort of electronictransmission of signals from a source to multiple receiving devices.Thus, a “broadcast” may be a cable broadcast, a satellite broadcast, aterrestrial broadcast, a traditional free television broadcast, a radiobroadcast, and/or an internet broadcast, and a “broadcaster” may be anyentity that transmits signals for reception by a plurality of receivingdevices. The signals may include content, (also referred to herein as“programs”), and/or commercials (also referred to herein as“advertisements”). An “advertiser” is any entity that provides anadvertisement for inclusion in a broadcast signal.

As is well known, advertisers pay significant sums of money tobroadcasters to have their commercials/advertisements broadcast inassociation with particular programs at particular times (i.e., during acertain “time slot”). An audience measurement entity (e.g., The NielsenCompany (US), LLC) can monitor media (e.g., commercials, televisionbroadcasts, movies, etc.) to which users are exposed. To monitor thebroadcasting of commercials, monitoring stations may be installed atvarious locations in various broadcasting areas. These monitoringstations collect information indicative of the different media (e.g.,commercials, television broadcasts, movies, etc.) broadcast in theirassociated broadcasting areas, the times/dates at which the media werebroadcast, and the stations or channels that broadcast those media. Thecollected information may be in an analog and/or a digital format. Theinformation collected for each piece of media identified by themonitoring stations may be some or all of the media as broadcast,signatures for some or all of the media as broadcast (e.g., a proxyuniquely representative of the content of the commercial), and/orwatermarks and/or other codes associated with, and possibly broadcastwith, the media. The collected information typically uniquely identifiesthe piece of media with which it is associated. It may also identify thestation broadcasting the media and/or the channel on which the media wasbroadcast and the time/date on which the media was broadcast.

Audio watermarking is a technique used to identify media such astelevision broadcasts, radio broadcasts, advertisements (televisionand/or radio), downloaded media, streaming media, prepackaged media,etc. Existing audio watermarking techniques identify media by embeddingone or more audio codes (e.g., one or more watermarks), such as mediaidentifying information and/or an identifier that may be mapped to mediaidentifying information, into an audio and/or video component. In someexamples, the audio or video component is selected to have a signalcharacteristic sufficient to hide the watermark. As used herein, theterms “code” or “watermark” are used interchangeably and are defined tomean any identification information (e.g., an identifier) that may beinserted or embedded in the audio or video of media (e.g., a program oradvertisement) for the purpose of identifying the media or for anotherpurpose such as tuning (e.g., a packet identifying header). As usedherein “media” refers to audio and/or visual (still or moving) contentand/or advertisements. To identify watermarked media, the watermark(s)are extracted and used to access a table of reference watermarks thatare mapped to media identifying information.

Unlike media monitoring techniques based on codes and/or watermarksincluded with and/or embedded in the monitored media, fingerprint orsignature-based media monitoring techniques generally use one or moreinherent characteristics of the monitored media during a monitoring timeinterval to generate a substantially unique proxy for the media. Such aproxy is referred to as a signature or fingerprint, and can take anyform (e.g., a series of digital values, a waveform, etc.) representativeof any aspect(s) of the media signal(s) (e.g., the audio and/or videosignals forming the media presentation being monitored). A signature maybe a series of signatures collected in series over a timer interval. Agood signature is repeatable when processing the same mediapresentation, but is relatively unique relative to other (e.g.,different) presentations of other (e.g., different) media. Accordingly,the term “fingerprint” and “signature” are used interchangeably hereinand are defined herein to mean a proxy for identifying media that isgenerated from one or more inherent characteristics of the media.

Signature-based media monitoring generally involves determining (e.g.,generating and/or collecting) signature(s) representative of a mediasignal (e.g., an audio signal and/or a video signal) output by amonitored media device and comparing the monitored signature(s) to oneor more reference signatures corresponding to known (e.g., reference)media sources. Various comparison criteria, such as a cross-correlationvalue, a Hamming distance, etc., can be evaluated to determine whether amonitored signature matches a particular reference signature. When amatch between the monitored signature and one of the referencesignatures is found, the monitored media can be identified ascorresponding to the particular reference media represented by thereference signature which matched the monitored signature. Becauseattributes, such as an identifier of the media, a presentation time, abroadcast channel, etc., are collected for the reference signature,these attributes may then be associated with the monitored media whosemonitored signature matched the reference signature. Example systems foridentifying media based on codes and/or signatures are long known andwere first disclosed in Thomas, U.S. Pat. No. 5,481,294, which is herebyincorporated by reference in its entirety.

In some examples, each monitoring station of an audience measuremententity is in communication with a central facility. The central facilityis typically provided with a database storing the audio/video codesand/or signatures that are used to identify known media. When a piece ofmedia is monitored in a broadcast, the central facility compares theaudio/video code and/or signature representative of the broadcast to thereference codes and/or reference signatures stored in the database toautomatically identify the piece of media. If a matching code and/orsignature is found in the database, the piece of media is identifiedfrom the database. The identity of the piece of media is then stored ina memory. The identity of the piece of media is typically stored inassociation with a time and/or a date stamp identifying the time/date atwhich the piece of media was broadcast and an identification of thechannel on which the piece of media was broadcast.

In some examples, in the event an identification of the media is notachieved (e.g., the media is not yet identified in the database because,for example, the video codes and/or signatures do not match anyreference video codes and/or signatures in the database), the monitoringstation determines that the media needs further processing to beidentified and monitored. In some examples, the central facility is ableto identify the media by matching audio codes/signatures representativeof the broadcast to the codes/signatures stored in the database.However, in some such examples, the video signatures of the broadcastmay not match the video signatures of the identified media. Such amismatch can occur during a split screen media presentation by amonitored media device. In such examples, prior monitoring stations maybe unable to reliably monitor the media because the video signatures ofthe broadcast do not match the video signatures of the identified media.

Example systems, methods, and apparatus disclosed herein include anaudio processor to process audio output from a media device to determineaudio metering data to identify first media presented by the mediadevice, a video signature generator to generate one or more videosignatures associated with video output from the media device, and acomputer vision processor to identify a first region-of-interest of asplit screen presentation of the media device including the first mediain response to a determination that the one or more generated videosignatures do not match reference video signatures corresponding to thefirst media.

In some examples, the audio metering data includes at least one of audiosignatures or audio watermarks. In some examples, a monitoring datacommunicator is to transmit audio metering data and the one or moregenerated video signatures to a central facility, and receive, from thecentral facility, an indication that the one or more generated videosignatures do not match reference video signatures. Examples disclosedherein include an image creator to capture one or more imagesrepresentative of the video output from the media device. In someexamples, the computer vision processor is to process the one or morecaptured images to identify candidate regions-of-interest, and to causethe video signature generator to generate respective video signaturesfor respective ones of the candidate regions-of-interest, the firstregion-of-interest corresponding to a matching region-of-interest havingvideo signatures corresponding to the first media.

In some examples, the computer vision processor is to determine that thefirst region-of-interest includes the first media in response toreceiving an indication from a central facility that the respectivevideo signatures for the first region-of-interest correspond to firstmedia. In some examples, the computer vision processor is to identifycandidate regions-of-interest using at least one process of edgedetection, line detection, feature detection, or feature extraction.Examples disclosed herein include an image variation detector to dividethe one or more captured images representative of the video output ofthe media device into a plurality of sub-regions, and determineinter-frame variation associated with each sub-region. Examplesdisclosed herein include an image sub-region encoder to create one ormore encoded synthetic images representative of the inter-framevariation associated with each sub-region. Examples disclosed hereinalso include a region-of-interest segmenter to segment the one or moresynthetic images into the candidate regions-of-interest.

A block diagram of an example media monitoring system 100 structured toidentify a piece of media in a split screen presentation provided via anexample media entertainment system 102 is illustrated in FIG. 1. Theexample media entertainment system 102 includes an example media source104, an example media device 106, and a display device 110. The examplemedia monitoring system 100 includes an example signal splitter 108, anexample meter 114, an example microphone sensor 107, an example camerasensor 109, and an example central facility 116. The example centralfacility 116 includes an example monitoring database 118. The componentsof the media entertainment system 102 and the media monitoring system100 may be connected in any appropriate manner including that shown inFIG. 1 in which the meter 114 is configured to monitor media exposureassociated with the media device 106 and/or display device 110. In someexamples, in a statistically selected household having one or more mediaentertainment systems 102, the meter 114 may be implemented as a singlehome unit and one or more site units. In such a configuration, thesingle home unit performs the functions of storing data and forwardingthe metering data to the central facility 116 for subsequent processingand/or storage. Each site unit is coupled to a corresponding mediaentertainment system 102 and performs the functions of collectingmetering data to the single home unit for that home. The home unitreceives and stores the metering data collected by the site units andsubsequently forwards that metering data to the central facility 116. Asanother example, some or all of the first example media monitoringsystem 100 can be implemented in a single media device, such as themedia device 106, the display device 110, a computer system, amultimedia computing device (e.g., such as a gaming device, a mobilephone, a personal digital assistant (PDA), etc.), etc.

The media source 104 may be any media source, such as a cable televisionprovider, a satellite television service provider, a direct satellitefeed, a radio frequency (RF) television service provider, an internetstreaming video/audio provider (e.g., such as Hulu, Inc.) avideo-on-demand (VOD) provider, a digital versatile disk (DVD) player, avideo cassette recorder (VCR), a video game console, a digital videorecorder (DVR), etc. The media source 104 may provide analog and/ordigital television signals to the media entertainment system 102, forexample, over a coaxial cable or via a wireless connection.

The media device 106 may be any multimedia device, such as a mobilephone, a set-top box, a gaming device, a personal digital assistant(PDA), a cable television converter, a direct broadcast satellite (DBS)decoder, a video cassette recorder (VCR), etc. In the illustratedexample of FIG. 1, the media device 106 is a set-top box that receives aplurality of broadcast channels from the media source 104. Typically,the media device 106 selects one of a plurality of broadcast channelsbased on a user input, and outputs one or more signals received via theselected broadcast channel. In the case of an analog signal, the mediadevice 106 tunes to a particular channel to obtain programming deliveredon that channel from the media source 104. For a digital signal, themedia device 106 may tune to a channel and decode certain packets ofdata to obtain programming delivered on a selected channel. For example,the media device 106 may tune to a major channel and then extract aprogram carried on a minor channel within the major channel via thedecoding process mentioned above. For some media entertainment systems102, for example, those in which the media source 104 is a standard RFanalog television service provider or a basic analog cable televisionservice provider, the media device 106 may not be present as itsfunction is performed by a tuner in the display device 110.

An output from the media device 106 is fed to a signal splitter 108,such as a single analog y-splitter in the case of an RF coaxialconnection between the media device 106 and the display device 110, anaudio/video splitter in the case of a direct audio/video connectionbetween the media device 106 and the display device 110, a digital datasplitter in the case of a digital data interface (e.g., such as ahigh-definition multimedia interface (HDMI)) used to connect the mediadevice 106 and the display device 110, etc. (For configurations in whichthe media device 106 is not present, the media source 104 may be coupleddirectly to the signal splitter 108 or the signal splitter 108 may bereplaced with a connection from an audio/video output of the displaydevice 110). In the example media entertainment system 102, the signalsplitter produces two signals indicative of the output from the mediadevice 106. Of course, a person of ordinary skill in the art willreadily appreciate that any number of signals may be produced by thesignal splitter 108.

In the illustrated example, one of the two signals from the signalsplitter 108 is fed to the display device 110 and the other signal isdelivered to the meter 114. The display device 110 may be any type oftelevision or television display device. For example, the display device110 may be a television or television display device that supports theNational Television Standards Committee (NTSC) standard, the PhaseAlternating Line (PAL) standard, Système Électronique pour Couleur avecMémoire (SECAM) standard, a standard developed by the AdvancedTelevision Systems Committee (ATSC), such as high definition television(HDTV), a standard developed by the Digital Video Broadcasting (DVB)Project, or may be a multimedia computer system, etc.

The second of the two signals from the signal splitter 108 (i.e., thesignal carried by connection 112 in FIG. 1) is coupled to an input ofthe meter 114. In an example implementation, the meter 114 implements adata logging and processing unit that may be used to generate viewingrecords and other viewing information useful for determining viewing andother metering information. Such an example meter 114 may collect a setof viewing records and transmit the collected viewing records to thecentral facility 116. The connection 112 may be a telephone line, areturn cable television connection, an RF or satellite connection, anInternet connection or the like. In some examples, the audio output fromthe media device 106 and/or the display device 110 is processed and/ormonitored by the microphone sensor 107. For example, the audio outputfrom the media device 106 and/or display device 110 may be detected bythe microphone sensor 107 and provided to the meter 114. In someexamples, the video output from the media device 106 and/or the displaydevice 110 is processed and/or monitored by the camera sensor 109. Forexample, the camera sensor 109 may be coupled with an image capturingdevice, such as a framegrabber, to capture images displaying on themedia device 106 and/or the display device 110. In some such examples,the camera sensor 109 may provide captured images from the video outputto the meter 114.

In the illustrated example, the meter 114 is configured to determineidentifying information corresponding to a piece of media being outputby the media device 106 during a split screen presentation. For example,a split screen presentation may include two or more regions of interestin which respective pieces of media are displayed simultaneously, whileonly one region of interest corresponds to the audio output from themedia device 106. The meter 114 in the illustrated example is configuredto identify, in combination with the central facility 116, which regionof interest of the split screen presentation corresponds to the piece ofmedia that corresponds to the audio output from the media device 106.

In an example implementation, the meter 114 may be configured togenerate audio signatures and/or video signatures and/or extract audioand/or video watermarks from the audio output and video output from themedia device 106 (received via connection 112) to identify the displayedmedia. In some examples, the meter 114 may generate audio and videosignatures for a single frame or successive frames of the broadcast fromthe media device 106. The meter 114 generates audio and/or videosignatures to identify the media that is being displayed. Monitoringreports of the identified media are sent by the meter 114 to the centralfacility 116 to be stored in a monitoring database 118. In otherexamples, the meter 114 may comprise the monitoring database 118 ofmonitoring reports. The meter 114 and the central facility 116 are incommunication to identify the piece of media displayed via a splitscreen presentation.

In some examples, the audio output from the media device 106 and/ordisplay device 110 is processed to detect audio code(s) (e.g., audiowatermark(s)) and/or generate audio signature(s) of the displayed media.The video output from the media device 106 and/or display device 110 isprocessed to generate video signature(s) of the displayed media. Whenthe video signature(s) do not match the media identified by the audiocode(s) and/or audio signature(s), the meter 114 determines that themedia device 106 is providing a split screen presentation. In someexamples, the meter 114 performs segmentation processing of thedisplayed media to identify a region-of-interest (ROI) of the splitscreen presentation that matches the media identified by the audiocode(s) and/or audio signature(s). In some examples, the meter 114implements any image segmentation technique or combination of techniquesto segment that displayed media into candidate ROI(s), and then themeter 114 generates video signature(s) for each candidate ROI. Thegenerated video signature(s) for each candidate ROI may be compared toreference video signature(s) of the reference media identified by theaudio code(s) and/or audio signature(s) to determine which candidateROI's video signature(s) match the video signature(s) of the identifiedmedia. In some examples, the matching candidate ROI is identified as thedetected ROI corresponding to the audio output from the media device106. In some examples, the meter 114 may store the location of thedetected ROI for use as a template to improve future matching times ofsplit screen presentations. In some examples, the segmentation may beimproved by performing segmentation on successive frames, comparing thesegmentations, and removing spurious segmentation lines fromconsideration. In some examples, the split screen media identificationmay be implemented without requiring a trigger condition of thegenerated video signature(s) not matching the media identified by theaudio code(s) and/or audio signature(s). In some such examples, themeter 114 may perform segmentation on the displayed media continuously.Split-screen media identification is disclosed in detail with connectionto FIGS. 2A-2B, 3.

A block diagram of an example implementation of the meter 114 of FIG. 1is illustrated in FIG. 2A. The example meter 114 includes an exampleaudio processor 202, an example video processor 204, and an examplesplit screen detector 206. The example audio processor 202 processes theaudio from an example audio input 208 configured to receive an audiosignal provided by a media device (e.g., such as the example mediadevice 106 and/or the example display device 110 of FIG. 1). Forexample, the audio signal received by the audio input 208 may correspondto audio obtained from the signal splitter 108 and/or the microphonesensor 107. The audio processor 202 processes the audio provided by themedia device 106 to obtain audio metering data for identification of themedia presented by the media device 106. In the illustrated example, theaudio processor 202 includes an example audio signature generator 210and/or an example watermark detector 212. As such, the audio meteringdata may include audio signatures and/or audio watermarks. The exampleaudio signature generator 210 generates one or more audio signaturescorresponding to the media. Any technique or combination of techniquesfor generating audio signatures may be implemented by the example audiosignature generator 210. Furthermore, although each audio signature maynot uniquely identify particular media, successive audio signaturesgenerated from the audio input together may form a set of audiosignatures that can uniquely represent a particular piece of media. Theexample watermark detector 212 extracts one or more watermarks embeddedin the audio of a piece of media using any appropriate technique orcombination of techniques. The example audio processor 202 sends thegenerated audio signature(s) and/or the extracted watermark(s)corresponding to the audio received by the audio input 208 to theexample monitoring data communicator 220 of the split screen detector206.

In the illustrated example, the video processor 204 includes an exampleimage creator 214 and an example video signature generator 216. Theexample image creator 214 accepts a video signal received via an examplevideo input 218. For example, the video signal received by the videoinput 218 may correspond to video obtained from the signal splitter 108and/or the camera sensor 109. In the illustrated example, the imagecreator 214 captures one or more images representative of the videooutput provided by the example media device 106 and/or the exampledisplay device 110. The captured image(s) from the image creator 214 isencoded with one or more properties/information that can aid in futureimage segmentation processes. The image creator 214 sends the capturedimage(s), which corresponds to media provided by a media device (e.g.,such as the example media device 106 of FIG. 1), to the example videosignature generator 216. The video signature generator 216 generates oneor more video signatures corresponding to media provided by the mediadevice. In the illustrated example, each video signature is generated byexamining the pixels in captured image. Any technique or combination oftechniques for generating video signatures may be implemented by theexample video signature generator 216. Furthermore, although eachsignature may not uniquely identify particular media, successive videosignatures generated by successive captured images from the imagecreator 214 together form a set of video signatures that can uniquelyrepresent particular media. The example video signature generator 216sends the generated video signature(s) and the captured image(s) to theexample monitoring data communicator 220.

The example monitoring data communicator 220 transmits the audiometering data (e.g., the generated audio signature(s) and/or detectedaudio watermark(s)) and video signature(s) to the example matchcommunicator 234 of the central facility 116 (see FIG. 2B) via anexample connection 222. The example connection 222 may be a wirelessInternet connection, a telephone line, a return path connection, an RFor satellite connection, etc. The processing performed by the centralfacility 116 is described in more detail below in connection with FIG.2B.

In the illustrated example of FIG. 2A, the monitoring data communicator220 receives an indication from the central facility 116 of whether ornot the generated video signature(s) substantially match the piece ofmedia identified by the audio metering data (e.g., the generated audiosignature(s) and/or the detected audio watermark(s). When the videosignature(s) representative of the media provided by the media device106 substantially match reference video signature(s) of the matchedreference media (i.e., the piece of media identified by the audiometering data), the central facility 116 transmits an indication to themonitoring data communicator 220 that the video signature(s) generatedfrom the video output by the media device 106 corresponds to the matchedreference media. However, if the central facility 116 transmits anindication to the monitoring data communicator 220 that the generatedvideo signature(s) do not match the video signature(s) of the matchedreference media, then the monitoring data communicator 220 may triggeran indication to the computer vision processor 224 that there was a lowvideo match score. In response, the example computer vision processor224 determines that the media provided by the media device 106 is in asplit screen presentation.

When a split screen presentation is determined, the captured image(s)from the image creator 214 are processed by the computer visionprocessor 224 using image segmentation and other processes to identify aregion of interest of the split screen presentation corresponding to thematched reference media associated with the output audio. The examplecomputer vision processor 224 includes an example image variationdetector 226, an example image sub-region encoder 228, an exampleregion-of-interest (ROI) segmenter 230, and an exampleregion-of-interest (ROI) selector 232. The example image variationdetector 226 determines inter-frame variation (e.g., difference and/ormotion) associated with each sub-region of the input captured imageobtained via the video input 218. For example, the image variationdetector 226 divides an input captured image into a plurality ofsub-regions, with each sub-region corresponding to a pixel or predefinedgrouping of pixels. The example image variation detector 226 thendetermines the inter-frame variation (e.g., difference and/or motion)associated with each sub-region of the input captured image. In anexample implementation, the example image variation detector 226determines such inter-frame variation by comparing the sub-regions ofthe current input captured image with the corresponding sub-regions ofone or more previous input captured image. Any appropriate imageprocessing technique for determining variation (e.g., difference ormotion) from successive capture images of a video presentation may beused to perform such a comparison. In another example implementation,the example image variation detector 226 determines the inter-framevariation of each sub-region of the captured image by extractingproperties/information indicative of inter-frame variation from theencoded video stream. Examples of such information include MPEG motionvectors that may be used to determine the motion associated with aparticular sub-region, and static macroblocks that may be used todetermine the absence of motion in a particular sub-region.

Either approach for inter-frame variation (e.g., difference and/ormotion) determination may be used depending upon the particularapplication. For example, the successive image frame comparison approachmay be used to determine inter-frame variation for analog or digitalmedia. The information/properties extraction approach is applicableprimarily to digital media, but does not require the buffering andcomparison of successive image frames.

Returning to FIG. 2A, the example image sub-region encoder 228 creates asynthetic image representative of the inter-frame variation (e.g.,difference and/or motion) associated with the sub-regions of the inputcaptured image as determined by the example image variation detector226. For example, the image sub-region encoder 228 takes the inter-framevariation determination made by the example image variation detector 226for each sub-region of the input captured image and encodes thedetermined variation (e.g., difference and/or motion) as a particularcolor and/or pattern in the respective sub-region of the determinedsynthetic image. The particular encoded color and/or pattern used for aparticular sub-region of the determined synthetic image depends upon thetype of variation detected in the respective sub-region of the inputcaptured image. For example, the example image sub-region encoder 228may represent motion to the left, right, up and down in a sub-region ofthe generated synthetic image by different first, second, third andfourth colors and/or patterns, respectively. Additionally, a fifth colorand/or pattern may be used to represent inter-frame differences notassociated with motion, such as differences associated with the abruptscene changes, gradual fading in and/or out of objects in a scene, etc.Additionally or alternatively, a sixth color and/or pattern may be usedto represent the absence of motion and/or substantially no inter-framedifference associated with the particular sub-region. The example ROIsegmenter 230 segments a synthetic image determined by the example imagesub-region encoder 228 into a plurality of candidate ROIs. In theillustrated example, the ROI segmenter 230 implements any appropriateedge detection, line detection, feature detection, feature extraction orsimilar technique, such as Canny edge detection, the generalized Houghtransform, etc., to detect edges in the synthetic image. Because thesynthetic image is made up of candidate ROIs each encoded to represent aparticular inter-frame variation (e.g., difference and/or motion), thedetected edges will correspond to boundaries between different types ofinter-frame variation. Accordingly, each segmented ROI of the syntheticimage corresponds to one or more connected sub-regions (e.g., pixels orgroup of pixels) that together exhibit one of the followingcharacteristics: (1) variation (e.g., motion) in a substantially uniformdirection, (2) substantially no variation, or (3) substantiallynon-uniform variation (e.g., substantially non-uniform differencesand/or motion) but which are bounded by regions exhibiting eithervariation (e.g., motion) in a substantially uniform direction orsubstantially no variation.

In some examples, the image variation detector 226 and the imagesub-region encoder 228 can be omitted from the computer vision processor224. In such examples, the ROI segmenter 230 receives the captured imagevia the video input 218. In response, the ROI segmenter 230 converts thecaptured image to a grayscale image using any appropriate technique orcombination of techniques. The ROI segmenter 230 segments the grayscaleimage into a plurality of ROIs. In some examples, the ROI segmenter 230implements any appropriate edge detection, line detection, featuredetection, feature extraction or similar technique, such as Canny edgedetection, the generalized Hough transform, etc., to detect edges in thegrayscale image.

In the illustrated example of FIG. 2A, the example ROI selector 232receives the set of candidate ROIs from the ROI segmenter 230. The ROIselector 232 generates respective sets of one or more video signaturescorresponding to each of the plurality of candidate ROIs. In theillustrated example, video signature(s) for a given candidate ROI aregenerated by examining the pixels in that ROI of the original imageframe from which the synthetic image was generated. Any technique forgenerating video signatures may be used to implement the example ROIselector 232. Furthermore, although each signature from a candidate ROImay not uniquely identify particular media in that candidate ROI of thecaptured image, successive video signatures generated by successivesynthetic images together form a set of video signatures that canuniquely represent particular media within that candidate ROI. Once theexample ROI selector 232 generates video signature(s) for each candidateROI of the original captured image, the ROI selector 232 transmits thecandidate ROI video signature(s) to the monitoring data communicator220. The example monitoring data communicator 220 transmits thecandidate ROI video signature(s) to the central facility 116. When thevideo signature(s) representative of a candidate ROI substantially matchreference video signature(s) of the matched reference media (i.e., thepiece of media identified by the audio metering data), the centralfacility 116 transmits an indication to the monitoring data communicator220 that the candidate ROI corresponds to the matched reference media.The monitoring data communicator 220 then transmits the indication tothe ROI selector 232, and the ROI selector 232 identifies the matchingcandidate ROI as the detected ROI corresponding to the audio output fromthe media device providing the media (e.g., the media device 106 fromFIG. 1). However, if the central facility 116 transmits an indication tothe monitoring data communicator 220 that no match is detected (i.e.,the video signatures for all candidate ROIs do not match the referencevideo signatures of the matched reference media), then the monitoringdata communicator 220 may trigger an indication to the computer visionprocessor 224 that there was a low video match score and the capturedimage should be re-segmented to create a new set of candidate ROIs. Thisre-segmentation process may be iterated until a match is found.

Once the video output is matched to the audio output by determining thedetected ROI, the meter 114 monitors the identified piece of media andthe example monitoring data communicator 220 may report monitoring datato the central facility 116 via connection 222 for record storage. Insome examples, the monitoring data may include audio metering data,video signature(s), identifying metadata, etc. In some examples, thesplit screen detector 206 saves the location of the detected ROI as atemplate to improve video signature matching times in future splitscreen orientation occurrences. For example, the split screen detector206 may save the location of the detected ROI along with identifyingmetadata (e.g., broadcast channel, a timestamp, a title of a televisionshow, etc.) to use as a template ROI for generating video signature(s)prior to conducting an image segmentation process.

In another example, the computer vision processor 224 performs imagesegmentation on media provided by a media device (e.g., the media device106 from FIG. 1) continuously. For example, the computer visionprocessor 224 may constantly segment the media and determine candidateROIs without the monitoring data communicator 220 triggeringsegmentation through an indication that there was a low video matchscore. In such examples, the computer vision processor 224 performsimage segmentation and ROI determination in the same manner as describedabove.

A block diagram of an example implementation of the central facility 116of FIG. 1 is illustrated in FIG. 2B. The example central facility 116includes the monitoring database 118, an example match communicator 234,an example media matcher 236, an example signature matcher 240 and anexample media database 238. In the illustrated example, the matchcommunicator 234 receives the audio metering data (e.g., audiowatermark(s) and/or audio signature(s)) and video signature(s)corresponding to the monitored media (i.e., the media presented by mediadevice 106) from the meter 114 via an example connection 222. Theexample match communicator 234 transmits the audio metering data to theexample media matcher 236. In the illustrated example, the media matcher236 is communicatively coupled to the media database 238. The examplemedia database 238 comprises audio/video reference signature(s) and/orwatermark(s) of a plurality of reference media (e.g., such as movies,television programs, commercials, promotional content, infomercials,public service announcements, etc.). In the illustrated example, whengenerated audio signature(s) and/or watermark(s) representative of themedia (i.e., media presented by media device 106) substantially matchreference audio signature(s) and/or watermark(s) (e.g., a high audiomatch score is identified), the example media matcher 236 identifies themedia as corresponding to the reference media represented by thematching reference signature(s) and/or watermark(s). (To improveidentification accuracy, multiple signatures and/or watermarksgenerated/detected from the audio input may be required to match acorresponding reference signature/watermark or set ofsignatures/watermarks before the media is identified as corresponding tothe matching reference media). In some examples, the media matcher 236may match the generated audio signature(s) and/or watermark(s) tomultiple reference media. For example, several commercials from the sameadvertiser may include the same audio output but have substantiallydifferent video output. In such examples, the media matcher 236 wouldmatch the generated audio signature(s) to a group of potential referencemedia with substantially the same audio output. The reference mediaidentified by the example media matcher 236 is associated withcorresponding audio and video signatures and/or watermarks. The examplemedia matcher 236 transmits the video signatures of the identifiedreference media to the example signature matcher 240. The example matchcommunicator 234 transmits the generated video signature(s)corresponding to media presented on media device 106 to the examplesignature matcher 240.

In the illustrated example of FIG. 2B, the signature matcher 240compares the video signatures of the identified reference media(received from the example media matcher 236) to the generated videosignature(s) reported by the meter 114 and corresponding to mediapresented on media device 106. In the illustrated example, when thegenerated video signatures reported by the meter 114 substantially match(e.g., a high video match score is identified by satisfying a high videoscore threshold) reference video signatures of the identified referencemedia, the example signature matcher 240 identifies the generated videosignatures reported by the meter 114 as corresponding to the matchedreference media. The example signature matcher 240 transmits identifyingmetadata corresponding to the identified media to the match communicator234, and the match communicator 234 transmits the match indication tothe meter 114 via connection 222. If no match is detected, the examplesignature matcher 240 may trigger an indication to the matchcommunicator 234 that there was a low video match score. The examplematch communicator 234 then transmits the indication to the meter 114via connection 222.

In some examples, the example match communicator 234 receives thecandidate ROI video signature(s) from the meter 114 via connection 222.The match communicator 234 transmits the candidate ROI videosignature(s) to the example signature matcher 240. The example signaturematcher 240 compares the candidate ROI video signature(s) to the videosignature(s) of the matched reference media (received previously fromthe example media matcher 236 matching the audio metering data). Ifvideo signatures representative of a candidate ROI substantially match(e.g., the video match score is high and exceeds a high threshold) thereference video signatures of the matched reference media, the signaturematcher 240 transmits an indication to the match communicator 234 thatthe candidate ROI corresponds to the matched reference media. Inresponse, the example match communicator 234 transmits the indication ofthe candidate ROI corresponding to the matched reference media to themeter 114 via connection 222. If none of the video signaturesrepresentative of the different candidate ROIs match the videosignatures of the matched reference media, then the example signaturematcher 240 may transmit an indication to the match communicator 234that there was a low video match score. In response, the example matchcommunicator 234 transmits the indication of the low video match scoreto the meter 114 via connection 222. This process may be iterated untila match is found. In some examples, the match communicator 234 of thecentral facility 116 receives a monitoring report from the meter 114corresponding to the monitoring of the matched media. In such examples,the central facility 116 may store the monitoring report in themonitoring database 118 of the illustrated example.

In some examples, the signature matcher 240 compares the candidate ROIvideo signatures(s) to the video signatures(s) of the identifiedreference media and an indeterminate match is found (e.g., the videomatch exceeds of otherwise satisfies an indeterminate match thresholdbut does not satisfy a full match threshold). In some such examples,when an indeterminate video match score is obtained, the signaturematcher 240 indicates that the media needs to be re-segmented resultingin a second plurality of candidate ROIs to be processed in the samemanner as described above.

While an example manner of implementing the media monitoring system 100of FIG. 1 is illustrated in FIGS. 2A-2B, one or more of the elements,processes and/or devices illustrated in FIG. 2A-2B may be combined,divided, re-arranged, omitted, eliminated and/or implemented in anyother way. Further, the example audio processor 202, the example videoprocessor 204, the example split screen detector 206, the example audiosignature generator 210, the example watermark detector 212, the exampleimage creator 214, the example video signature generator 216, theexample monitoring data communicator 220, the example computer visionprocessor 224, the example image variation detector 226, the exampleimage sub-region encoder 228, the example ROI segmenter 230, the exampleROI selector 232, the example match communicator 234, the example mediamatcher 236, the example signature matcher 240 and/or, more generally,the example media monitoring system 100 of FIG. 1 may be implemented byhardware, software, firmware and/or any combination of hardware,software and/or firmware. Thus, for example, any of the example audioprocessor 202, the example video processor 204, the example split screendetector 206, the example audio signature generator 210, the examplewatermark detector 212, the example image creator 214, the example videosignature generator 216, the example monitoring data communicator 220,the example computer vision processor 224, the example image variationdetector 226, the example image sub-region encoder 228, the example ROIsegmenter 230, the example ROI selector 232, the example matchcommunicator 234, the example media matcher 236, the example signaturematcher 240 and/or, more generally, the example media monitoring system100 of FIG. 1 could be implemented by one or more analog or digitalcircuit(s), logic circuits, programmable processor(s), programmablecontroller(s), graphics processing unit(s) (GPU(s)), digital signalprocessor(s) (DSP(s)), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)). When reading any of theapparatus or system claims of this patent to cover a purely softwareand/or firmware implementation, at least one of the example, audioprocessor 202, the example video processor 204, the example split screendetector 206, the example audio signature generator 210, the examplewatermark detector 212, the example image creator 214, the example videosignature generator 216, the example monitoring data communicator 220,the example computer vision processor 224, the example image variationdetector 226, the example image sub-region encoder 228, the example ROIsegmenter 230, the example ROI selector 232, the example matchcommunicator 234, the example media matcher 236, the example signaturematcher 240 is/are hereby expressly defined to include a non-transitorycomputer readable storage device or storage disk such as a memory, adigital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc.including the software and/or firmware. Further still, the example mediamonitoring system 100 of FIG. 1 may include one or more elements,processes and/or devices in addition to, or instead of, thoseillustrated in FIGS. 2A-2B, and/or may include more than one of any orall of the illustrated elements, processes and devices. As used herein,the phrase “in communication,” including variations thereof, encompassesdirect communication and/or indirect communication through one or moreintermediary components, and does not require direct physical (e.g.,wired) communication and/or constant communication, but ratheradditionally includes selective communication at periodic intervals,scheduled intervals, aperiodic intervals, and/or one-time events.

A flowchart representative of example hardware logic or machine readableinstructions for implementing the media monitoring system 100 of FIGS.1, 2A-2B is shown in FIGS. 3A-3B, 4, 5. The machine readableinstructions may be a program or portion of a program for execution by aprocessor such as the processor 612 shown in the example processorplatform 600 discussed below in connection with FIG. 6. The program maybe embodied in software stored on a non-transitory computer readablestorage medium such as a CD-ROM, a floppy disk, a hard drive, a DVD, aBlu-ray disk, or a memory associated with the processor 612, but theentire program and/or parts thereof could alternatively be executed by adevice other than the processor 612 and/or embodied in firmware ordedicated hardware. Further, although the example program is describedwith reference to the flowchart illustrated in FIGS. 3A-3B, 4, 5, manyother methods of implementing the example media monitoring system 100may alternatively be used. For example, the order of execution of theblocks may be changed, and/or some of the blocks described may bechanged, eliminated, or combined. Additionally or alternatively, any orall of the blocks may be implemented by one or more hardware circuits(e.g., discrete and/or integrated analog and/or digital circuitry, anFPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logiccircuit, etc.) structured to perform the corresponding operation withoutexecuting software or firmware.

As mentioned above, the example processes of FIGS. 1, 2A-2B may beimplemented using executable instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media.

“Including” and “comprising” (and all forms and tenses thereof) are usedherein to be open ended terms. Thus, whenever a claim employs any formof “include” or “comprise” (e.g., comprises, includes, comprising,including, having, etc.) as a preamble or within a claim recitation ofany kind, it is to be understood that additional elements, terms, etc.may be present without falling outside the scope of the correspondingclaim or recitation. As used herein, when the phrase “at least” is usedas the transition term in, for example, a preamble of a claim, it isopen-ended in the same manner as the term “comprising” and “including”are open ended. The term “and/or” when used, for example, in a form suchas A, B, and/or C refers to any combination or subset of A, B, C such as(1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, and(6) B with C.

FIGS. 3A-3B are example flow diagrams representative of machine readableinstructions 300 that may be executed to implement the meter 114 of themedia monitoring system 100 of FIGS. 1, 2A-2B. The example processbegins at block 302, wherein the audio processor 202 determines audiomonitoring data (e.g., audio signature(s), audio watermark(s), etc.) toidentify media corresponding to audio output from a media device (e.g.,the media device 106 of FIG. 1). The image creator 214 of the videoprocessor 204 captures one or more images representative of a videooutput from the media device (block 304). At block 306, the videosignature generator 216 of the video processor 204 generates one or morevideo signature(s) corresponding to the media from the one or morecaptured images. The monitoring data communicator 220 then transmits thegenerated video signature(s) and audio monitoring data to the centralfacility 116 (block 308).

If, at block 310, the monitoring data communicator 220 receives anindication from the central facility 116 that the reference videosignature(s) representative of reference media corresponding to theaudio output from the media device match the generated videosignature(s), then control proceeds to block 324 where the meter 114monitors the matched media and the monitoring data communicator 220reports a monitoring report to the central facility 116. If, at block310, the monitoring data communicator 220 receives an indication fromthe central facility 116 that the reference video signature(s)representative of reference media corresponding to the audio output fromthe media device do not match the generated video signature(s), thencontrol proceeds to block 312. At block 312, the monitoring datacommunicator 220 triggers an indication to the computer vision processor224 that the video match score is low, and the computer vision processor224 determines that the media device 106 is in a split screenpresentation. At block 314, the example computer vision processor 224 ofthe split screen detector 206 processes the captured image to identifycandidate regions-of-interest (ROIs). An example process that can beused to implement the operation of block 314 is described below inconnection with FIG. 4.

At block 316, the example video signature generator 216 of the examplevideo processor 204 generates one or more video signatures for eachcandidate ROI. Then, at block 318, the monitoring data communicator 220transmits the generated video signature(s) for each candidate ROI to thecentral facility 116. If, at block 320, the monitoring data communicator220 receives an indication from the central facility 116 that thereference video signature(s) do not match the generated videosignature(s) of one of the candidate ROIs, then control proceeds back toblock 316 to generate new video signature(s) for each candidate ROI. If,at block 320, the monitoring data communicator 220 receives anindication from the central facility 116 that the reference videosignature(s) match the generated video signature(s) of one of thecandidate ROIs, then control proceeds to block 322. At block 322, theROI selector 232 determines the matching candidate ROI as the detectedROI corresponding to the audio output from the media device. At block324, the meter 114 monitors the matched media and the example monitoringdata communicator 220 reports a monitoring report to the centralfacility 116. The example process concludes.

FIG. 4 is an example flow diagram representative of machine readableinstructions 400 that may be executed to implement the operation ofblock 314 of FIG. 3B. The example process begins at block 402 whereinthe example image variation detector 226 of the computer visionprocessor 224 divides the captured image(s) into a plurality ofsub-regions. At block 404, the example image variation detector 226determines the inter-frame variation associated with each of thesub-regions. Any appropriate image processing technique for determininginter-frame variation from successive capture images of a videopresentation may be used to perform such a determination. The exampleimage sub-region encoder 228 of the computer vision processor 224creates encoded synthetic image(s) representative of the inter-framevariation associated with each sub-region (block 406). At block 408, theexample ROI segmenter 230 of the computer vision processor 224 segmentsthe synthetic image(s) into a plurality of candidate regions-of-interest(ROIs) and the process concludes.

FIG. 5 is an example flow diagram representative of machine readableinstructions 500 that may be executed to implement the central facility116 of the media monitoring system 100 of FIGS. 1, 2A-2B. The exampleprocess begins at block 502 where the example match communicator 234receives the generated audio metering data and video signature(s) fromthe meter 114. At block 504, the example media matcher 236 identifiesthe media corresponding to the audio output by matching the generatedaudio metering data to reference audio in an example media database 238.The example signature matcher 240 compares the reference videosignature(s) of the matched reference media to the generated videosignature(s) (block 506). If, at block 508, the signature matcher 240determines that the reference video signature(s) match the generatedvideo signature(s), then control proceeds to block 518 in which theexample match communicator 234 transmits an indication to the meter 114that a match was found. If, at block 508, the signature matcher 240determines that the reference video signature(s) do not match thegenerated video signature(s), then control proceeds to block 510.

At block 510, the example match communicator 234 transmits an indicationto the meter 114 that a match was not found. At block 512, the examplematch communicator 234 receives the generated video signature(s) foreach candidate ROI from the meter 114. The signature matcher 240 thencompares the reference video signature(s) of the matched reference mediato the generated video signature(s) of a candidate ROI (block 514). If,at block 516, the example signature matcher 240 determines that thereference video signature(s) do not match the generated videosignature(s) of one of the candidate ROIs, then control proceeds back toblock 514 to compare the reference video signature(s) of the identifiedreference media to the generated video signature(s) of a new candidateROI. If, at block 516, the example signature matcher 240 determines thatthe reference video signature(s) match the generated video signature(s)of one of the candidate ROIs, then control proceeds to block 520. Atblock 520, the example match communicator 234 transmits an indication tothe meter 114 identifying a matching candidate ROI. The example matchcommunicator 234 receives a monitoring report of the monitoredidentified media from the meter 114 (block 520). At block 522, theexample monitoring database 118 stores the monitoring report of themonitored matched media and the process concludes.

FIG. 6 is a block diagram of an example processor platform 600structured to execute the instructions of FIGS. 3A-B, 4 to implement themeter 114 of FIGS. 1, 2A. The processor platform 600 can be, forexample, a server, a personal computer, a workstation, a self-learningmachine (e.g., a neural network), a mobile device (e.g., a cell phone, asmart phone, a tablet such as an iPad™), a personal digital assistant(PDA), an Internet appliance, a DVD player, a CD player, a digital videorecorder, a Blu-ray player, a gaming console, a personal video recorder,a set top box, a headset or other wearable device, or any other type ofcomputing device.

The processor platform 600 of the illustrated example includes aprocessor 612. The processor 612 of the illustrated example is hardware.For example, the processor 612 can be implemented by one or moreintegrated circuits, logic circuits, microprocessors, GPUs, DSPs, orcontrollers from any desired family or manufacturer. The hardwareprocessor may be a semiconductor based (e.g., silicon based) device. Inthis example, the processor implements the example audio processor 202,the example video processor 204, the example split screen detector 206,the example audio signature generator 210, the example watermarkdetector 212, the example image creator 214, the example video signaturegenerator 216, the example monitoring data communicator 220, the examplecomputer vision processor 224, the example image variation detector 226,the example image sub-region encoder 228, the example ROI segmenter 230,and the example ROI selector 232.

The processor 612 of the illustrated example includes a local memory 613(e.g., a cache). The processor 612 of the illustrated example is incommunication with a main memory including a volatile memory 614 and anon-volatile memory 616 via a bus 618. The volatile memory 614 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory(RDRAM®) and/or any other type of random access memory device. Thenon-volatile memory 616 may be implemented by flash memory and/or anyother desired type of memory device. Access to the main memory 614, 616is controlled by a memory controller.

The processor platform 600 of the illustrated example also includes aninterface circuit 620. The interface circuit 620 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), a Bluetooth® interface, a near fieldcommunication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices 622 are connectedto the interface circuit 620. The input device(s) 622 permit(s) a userto enter data and/or commands into the processor 712. The inputdevice(s) can be implemented by, for example, an audio sensor, amicrophone, a camera (still or video), a keyboard, a button, a mouse, atouchscreen, a track-pad, a trackball, isopoint and/or a voicerecognition system.

One or more output devices 624 are also connected to the interfacecircuit 620 of the illustrated example. The output devices 724 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay (LCD), a cathode ray tube display (CRT), an in-place switching(IPS) display, a touchscreen, etc.), a tactile output device, a printerand/or speaker. The interface circuit 620 of the illustrated example,thus, typically includes a graphics driver card, a graphics driver chipand/or a graphics driver processor.

The interface circuit 620 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem, a residential gateway, a wireless access point, and/or a networkinterface to facilitate exchange of data with external machines (e.g.,computing devices of any kind) via a network 626. The communication canbe via, for example, an Ethernet connection, a digital subscriber line(DSL) connection, a telephone line connection, a coaxial cable system, asatellite system, a line-of-site wireless system, a cellular telephonesystem, etc.

The processor platform 600 of the illustrated example also includes oneor more mass storage devices 628 for storing software and/or data.Examples of such mass storage devices 628 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, redundantarray of independent disks (RAID) systems, and digital versatile disk(DVD) drives.

The machine executable instructions 632 of FIGS. 3A-3B, 4 may be storedin the mass storage device 628, in the volatile memory 614, in thenon-volatile memory 616, and/or on a removable non-transitory computerreadable storage medium such as a CD or DVD.

FIG. 7 is a block diagram of an example processor platform 700structured to execute the instructions of FIG. 5 to implement thecentral facility 116 of FIGS. 1, 2B. The processor platform 700 can be,for example, a server, a personal computer, a workstation, aself-learning machine (e.g., a neural network), a mobile device (e.g., acell phone, a smart phone, a tablet such as an iPad™), a personaldigital assistant (PDA), an Internet appliance, a DVD player, a CDplayer, a digital video recorder, a Blu-ray player, a gaming console, apersonal video recorder, a set top box, a headset or other wearabledevice, or any other type of computing device.

The processor platform 700 of the illustrated example includes aprocessor 712. The processor 712 of the illustrated example is hardware.For example, the processor 712 can be implemented by one or moreintegrated circuits, logic circuits, microprocessors, GPUs, DSPs, orcontrollers from any desired family or manufacturer. The hardwareprocessor may be a semiconductor based (e.g., silicon based) device. Inthis example, the processor implements the example match communicator234, the example media matcher 236, and the example signature matcher240.

The processor 712 of the illustrated example includes a local memory 713(e.g., a cache). The processor 712 of the illustrated example is incommunication with a main memory including a volatile memory 714 and anon-volatile memory 716 via a bus 718. The volatile memory 714 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory(RDRAM®) and/or any other type of random access memory device. Thenon-volatile memory 716 may be implemented by flash memory and/or anyother desired type of memory device. Access to the main memory 714, 716is controlled by a memory controller.

The processor platform 700 of the illustrated example also includes aninterface circuit 720. The interface circuit 720 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), a Bluetooth® interface, a near fieldcommunication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices 722 are connectedto the interface circuit 720. The input device(s) 722 permit(s) a userto enter data and/or commands into the processor 712. The inputdevice(s) can be implemented by, for example, an audio sensor, amicrophone, a camera (still or video), a keyboard, a button, a mouse, atouchscreen, a track-pad, a trackball, isopoint and/or a voicerecognition system.

One or more output devices 724 are also connected to the interfacecircuit 720 of the illustrated example. The output devices 724 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay (LCD), a cathode ray tube display (CRT), an in-place switching(IPS) display, a touchscreen, etc.), a tactile output device, a printerand/or speaker. The interface circuit 720 of the illustrated example,thus, typically includes a graphics driver card, a graphics driver chipand/or a graphics driver processor.

The interface circuit 720 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem, a residential gateway, a wireless access point, and/or a networkinterface to facilitate exchange of data with external machines (e.g.,computing devices of any kind) via a network 726. The communication canbe via, for example, an Ethernet connection, a digital subscriber line(DSL) connection, a telephone line connection, a coaxial cable system, asatellite system, a line-of-site wireless system, a cellular telephonesystem, etc.

The processor platform 700 of the illustrated example also includes oneor more mass storage devices 728 for storing software and/or data.Examples of such mass storage devices 728 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, redundantarray of independent disks (RAID) systems, and digital versatile disk(DVD) drives.

The machine executable instructions 732 of FIG. 5 may be stored in themass storage device 728, in the volatile memory 714, in the non-volatilememory 716, and/or on a removable non-transitory computer readablestorage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods,apparatus and articles of manufacture have been disclosed that improvethe accuracy of media monitoring by detecting when a media device isproviding a split screen presentation. Prior media monitoring units maynot be capable of detecting when a media device is in a split screenorientation. In some instances, a media device may be displaying two ormore pieces of media simultaneously in a split screen presentation. Insuch examples, the audio output from the media device may correspond toonly one of the respective pieces of media in the split screenpresentation. It is in the interest of audience monitoring entities,advertisement companies, broadcast networks, etc. to (1) detect when amedia device is experiencing a split screen presentation, and (2) beable to identify which piece of media corresponds to the audio outputfrom the media device. This allows an audience measurement entity tocorrectly attribute the monitoring data to the respective media.Examples disclosed herein are able to solve the inherently technicalproblem of detecting when a media device is providing a split screenpresentation.

Although certain example methods, apparatus and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. An apparatus comprising: memory; and at least oneprocessor to execute computer readable instructions to: generate one ormore video signatures associated with video output from a media device;determine whether the one or more video signatures correspond to firstmedia identified based on audio output from the media device; and inresponse to a determination that the one or more generated videosignatures do not correspond to the first media: determine that thevideo corresponds to a split-screen presentation having a plurality ofregions-of-interest; and identify a first one of the regions-of-interestof the split-screen presentation that includes the first media.
 2. Theapparatus of claim 1, wherein the at least one processor is to transmitthe generated video signatures to a central facility.
 3. The apparatusof claim 2, wherein the at least one processor is to obtain, from thecentral facility, an indication that the one or more generated videosignatures do not match reference video signatures.
 4. The apparatus ofclaim 1, wherein the at least one processor is to: capture one or moreimages representative of the video output from the media device; processthe one or more captured images to identify candidateregions-of-interest; and generate respective video signatures forrespective ones of the candidate regions-of-interest, a firstregion-of-interest corresponding to a matching region-of-interest havingvideo signatures corresponding to the first media.
 5. The apparatus ofclaim 4, wherein the at least one processor is to determine that thefirst region-of-interest includes the first media in response to anindication from a central facility that the respective video signaturesfor the first region-of-interest correspond to the first media.
 6. Theapparatus of claim 4, wherein the at least one processor is to identifythe candidate regions-of-interest using at least one process of edgedetection, line detection, feature detection, or feature extraction. 7.The apparatus of claim 4, wherein the at least one processor is todivide the one or more captured images representative of the videooutput of the media device into a plurality of sub-regions and determineinter-frame variation associated with each sub-region.
 8. The apparatusof claim 7, wherein the at least one processor is to create one or moreencoded synthetic images representative of the inter-frame variationassociated with each sub-region.
 9. The apparatus of claim 8, whereinthe at least one processor is to segment the one or more syntheticimages into candidate regions-of-interest.
 10. A method comprising:generating one or more video signatures associated with video outputfrom a media device; determining whether the one or more videosignatures correspond to first media identified based on audio outputfrom the media device; and in response to a determination that the oneor more generated video signatures do not correspond to the first media:determining that the video corresponds to a split-screen presentationhaving a plurality of regions-of-interest; and identifying a first oneof the regions-of-interest of the split-screen presentation thatincludes the first media.
 11. The method of claim 10, further including:transmitting the generated video signatures to a central facility; andreceiving, from the central facility, an indication that the one or moregenerated video signatures do not match reference video signatures. 12.The method of claim 10, further including: capturing one or more imagesrepresentative of the video output from the media device; processing theone or more captured images to identify candidate regions-of-interest,and generating video signatures for respective ones of the candidateregions-of-interest, a first region-of-interest corresponding to amatching region-of-interest having video signatures corresponding to thefirst media.
 13. The method of claim 12, further including identifyingthe first region-of-interest in response to an indication from a centralfacility that the respective video signatures for a first candidateregion-of-interest of the candidate regions-of-interest correspond tothe first media.
 14. The method of claim 12, wherein the identifying ofthe candidate regions-of-interest is based on at least one of edgedetection, line detection, feature detection, or feature extraction. 15.The method of claim 12, further including: dividing the one or morecaptured images representative of the video output of the media deviceinto a plurality of sub-regions; determining inter-frame variationassociated with each sub-region; creating one or more encoded syntheticimages representative of the inter-frame variation associated with eachsub-region; and segmenting the one or more synthetic images intocandidate regions-of-interest.
 16. A non-transitory computer readablestorage medium comprising instructions that, when executed, cause amachine to at least: generate one or more video signatures associatedwith video output from a media device; determine whether the one or morevideo signatures correspond to first media identified based on audiooutput from the media device; and in response to a determination thatthe one or more generated video signatures do not correspond to thefirst media: determine that the video corresponds to a split-screenpresentation having a plurality of regions-of-interest; and identify afirst one of the regions-of-interest of the split-screen presentationthat includes the first media.
 17. The non-transitory computer readablestorage medium as defined in claim 16, wherein the instructions causethe machine to: transmit audio metering data and the generated videosignatures to a central facility; and obtain, from the central facility,an indication that the video signatures do not match reference videosignatures.
 18. The non-transitory computer readable storage medium asdefined in claim 16, wherein the instructions cause the machine to:capture one or more images representative of the video output from themedia device; process the one or more captured images to identifycandidate regions-of-interest; and generate video signatures forrespective ones of the candidate regions-of-interest, a firstregion-of-interest corresponding to a matching region-of-interest havingvideo signatures corresponding to the first media.
 19. Thenon-transitory computer readable storage medium of claim 18, wherein theinstructions cause the machine to identify the first region-of-interestin response to an indication from a central facility that the respectivevideo signatures for a first candidate region-of-interest of thecandidate regions-of-interest correspond to the first media.
 20. Thenon-transitory computer readable storage medium as defined in claim 18,wherein the instructions cause the machine to: divide the one or morecaptured images representative of the video output of the media deviceinto a plurality of sub-regions; determine inter-frame variationassociated with each sub-region; create one or more encoded syntheticimages representative of the inter-frame variation associated with eachsub-region; and segment the one or more synthetic images into candidateregions-of-interest.